The exemplary embodiment relates to image segmentation and finds particular application in connection with a system and method which uses segmentation information from similar images to segment a selected image.
Image segmentation refers to the partitioning of an image into two or more regions, typically corresponding to different semantic concepts. For example, a photographic image of a beach scene may be partitioned into sand, sea, and sky areas, or a document image may be partitioned into background, pictures, tables, and text. In some cases, segmentation is used for object detection, where the aim is to segment the image into a foreground region, which corresponds to the object of interest, and a background region, which corresponds to the rest of the image. The detected object can then be extracted from the image or a decision made based on the detection.
Automated image segmentation is a useful component of many image-related business processes. For example, photographs of vehicles may be subjected to segmentation techniques to identify the region of the image which corresponds to a license plate. OCR techniques may then be applied to this region to identify the license number or to see if it matches another license plate.
Existing segmentation techniques are based on heuristics which exploit the a priori known characteristics of the object to be segmented, such as characteristics of text. For example, some exploit the frequent presence of horizontal and vertical edges. See, for example, Wonder Alves, et al., “Text localization in scene images by morphological filters,” in SIBGRAPI, 2009, and Toan Dinh Nguyen, et al., “Tensor voting based text localization in natural scene images,” IEEE Signal Processing Letters, 17, July 2010. Others rely on high local contrast or constant stroke width. See, for example, Paolo Comelli, et al., “Optical recognition of motor vehicle license plates.” IEEE Trans. on VT, 44, November 1995; Boris Epshtein, et al., “Detecting text in natural scenes with stroke width transform,” in CVPR, pages 2963-2970, 2010. These techniques have very narrow applicability, since the prior knowledge of the images of interest is incorporated into the software, and therefore such methods do not generalize well to other segmentation tasks.
Data-driven approaches are more general. A common approach of this type is to extract local descriptors (patches) from images. For example gradient-based keypoint descriptors are extracted at the locations of a dense, regular grid. These can be employed to train classifiers that will estimate the region class for the patches of a new image.
In object detection, a classifier is trained on positive examples, corresponding to sub-images containing the segmented objects of interest, and negative examples, corresponding to sub-images not containing the object. Such a classifier can be used to score all the possible sub-regions of a new image to identify the most probable location of the object.
Such methods resort to multiple local classification steps in order to obtain the segmentation map and can thus be computationally expensive.
The exemplary embodiment provides a segmentation method which enables object detection and other segmentation tasks to be performed based on a global representation of an image and a set of images for which segmentation information is available.